Factor Analysis Using Batch and Online EM

A simple way to encode input patterns is to suppose that each input can be well-approximated by a linear combination of component vectors, where the amplitudes of the vectors are modulated to match the input. For a given training set, the most appropriate set of component vectors will depend on how we expect the modulation levels to behave and how we measure the distance between the input and its approximation. These effects can be captured by a generative probability model that specifies a distribution p(z) over modulation levels z = (z1; : : : ; zK) and a distribution p(xjz) over sensors x = (x1; : : : ; xN) given the modulation levels. The linear combination constraint is given by E[xjz]= Λz; (1)